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ABSTRACT 


This  research  memorandum  evaluates  the  use  of 
Video  Telctraining  (VTT)  to  deliver  Navy  F-school 
instruction  to  students  at  remote  sites.  Using  data 
collected  from  a  four-site,  fully  interactive  audio-video 
network,  the  analysis  focuses  on  system  utilization, 
training  effectiveness,  downtime,  and  savings  to  the 
Navy. 


EXECUTIVE  SUMMARY 


Video  Teletraining  (VTT)  is  a  method  of  delivering  Navy  schoolhouse  instruction  to 
students  at  remote  sites.  VTT  involves  the  use  of  a  full  duplex  audio-visual  nuwork  between 
existing  schoolhouses  and  remote  locations.  The  network  is  fully  interactive  in  that  participants 
at  different  locations  can  talk  to  and  see  one  another  in  real  time. 

In  March  1989,  the  Navy  established  a  multipoint  secure  VTT  system  with  sites  at 
Charleston,  Dam  Neck,  Mayport,  and  Norfolk.  All  sites  are  interconnected  via  commercial 
satellite.  The  Center  for  Naval  Analyses  was  tasked  by  Commander,  Naval  Education  and 
Training  (CNET)  to  evaluate  this  system  during  its  first  six  months  of  operation.  Results  from 
this  analysis  can  be  grouped  into  four  areas: 

•  System  utilization 

•  Training  effectiveness 

•  Downtime 

•  Net  savings  to  the  Navy. 

SYSTEM  UTILIZATION 

The  VTT  network  consisted  of  six  classrooms.  Dam  Neck  and  Charleston  each  had 
two  classrooms,  and  Norfolk  and  Mayport  each  had  one  classroom.  On  the  average  weekday, 
46  percent  of  the  classrooms  were  being  used  for  either  training  or  holding  conferences.  The 
principal  reason  for  the  low  overall  utilization  rate  was  that  the  second  classrooms  at  both 
Dam  Neck  and  Charleston  were  typically  unused.  This  problem  has  recently  been  addressed  by 
eliminating  the  second  classrooms  at  both  of  these  sites  and  establishing  a  new  site  at  Newport 

VTT  was  employed  to  train  705  students  in  25  convenings  of  12  F-school  courses.  In 
addition,  the  system  was  used  to  support  Navy  training  exercises,  conferences,  and  Office  of 
Civilian  Personnel  Management  (OCPM)  training.  The  total  attendance  from  these  other  uses  of 
the  system  was  737  persons,  which  was  achieved  using  only  16  convenings.  Relative  to 
VTT  course  training,  the  other  uses  had  much  higher  throughput  per  convening  because  they 
typically  used  more  sites  per  convening  and  they  had  higher  attendance  per  site. 

TRAINING  EFFECTIVENESS 

Course  grades  of  students  at  the  remote  sites  were  compared  to  those  of  students  at  the 
originating  site  to  determine  how  effective  the  VTT  method  of  instruction  is  relative  to  the 
traditional  method  of  instruction.  Students  at  the  remote  sites  serve  as  the  test  group  in  the 
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evaluation  because  they  could  see,  hear,  and  speak,  with  the  instructor  exclusively  through  the 
VI T  technology.  In  contrast,  students  in  the  same  room  as  the  instructor  (i.e,,  at  die  originating 
site)  serve  as  the  control  group  because  they  interacted  with  the  instructor  without  using  die 
VTT  technology.  Although  students  at  the  originating  site  are  the  best  available  control  group, 
they  are  not  a  perfect  control  group  since  some  of  their  learning  (e.g..  graphics  on  television 
monitor)  did  rely  on  the  VTT  technology. 

The  average  grade  across  all  sites  was  87.2  on  a  scale  of  0  to  100.  Controlling  for  a 
student’s  mental  aptitude  and  experience  in  the  Navy,  the  number  of  sites,  and  differences  in 
grading  across  courses,  a  regression  analysis  found  diat  grades  at  remote  sites  were,  on  average, 
2.4  points  lower  than  grades  at  the  originaung  site.  This  suggests  that  the  average  grade  under 
VTT  would  be  lower  than  the  average  grade  under  traditional  methods  of  instruction.  The 
difference  in  grades  was  small  enough  that  failure,  rates  (i.e..  percentage  of  students  with  grades 
less  than  70)  did  not  differ  significantly  between  the  two  training  methods.  Since  the 
VTT  Steering  Committee  has  selected  the  difference  in  failure  rates  as  the  relevant  measure  of 
training  effectiveness,  the  difference  in  grades  is  not  considered  to  be  practically  significant. 
However,  it  should  be  noted  that  tnesc  courses  typically  have  very  low  failure  rates.  If  VTT  was 
used  in  courses  with  relatively  high  failure  rates,  the  difference  in  grades  could  become  prac¬ 
tically  significant. 

In  a  survey  of  VTT  students,  64.2  percent  of  those  at  remote  sites  preferred  traditional 
methods  of  instruction  to  VTT.  The  survey  also  identified  three  main  areas  of  deficiency  in  the 
VTT  method  of  instruction:  the  quality  of  the  video,  the  level  of  instructor-student  interaction, 
and  die  quality  of  he  audio.  improving  the  quality  of  the  audio  and  video  transmissions  is 
primarily  a  technical  issue.  During  die  latter  part  of  the  evaluation  period,  efforts  were  made  to 
improve  VTT  by  increasing  the  size  of  the  television  monitors,  adding  additional  microphones 
for  the  students,  using  a  cordless  microphone  for  the  instructor,  and  incorporating  new  software 
in  video  comprcssion/decomprcssion.  Since  these  courses  were  not  designed  for  VTT,  im¬ 
proving  instructor-student  interaction  will  require  training  of  instructors  in  effectively  use  the 
medium  and  redesigning  course  materials  to  better  fit  die  medium. 

Based  on  a  regression  analysis,  when  die  total  number  of  sites  increased  from  two  to  three, 
die  average  grade  dropped  2.2  points  at  both  originating  and  remote  sites.  The  associated 
increase  in  the  total  number  of  students  accounted  for  only  18  percent  of  this  decline.  Therefore, 
die  number  of  sites  was  a  more  important  determinant  of  course  grades  than  the  number  of 
students.  Whether  grades  would  remain  die  same  or  further  deteriorate  with  an  increase  from 
two  remote  sites  to  three  or  more  remote  sites,  is  an  important  question  lor  limber  research. 

DOWNTIME 

When  the  transmission  is  degraded  to  a  large  degree  or  lost  altogether  in  a  VTT  course,  die 
course  must  stop  and  the  time  dial  students  must  wail  for  it  to  restan  is  classified  as  downtime. 
During  die  first  six  months,  the  VTT  system  was  down  rather  infrequently,  with  an  average  of 
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1.4  incidents  per  course  convening.  The  average  downtime  per  incident  was  29  minutes. 
Equipment  problems  were  the  most  frequent  cause  of  downtime;  they  accounted  for  half  of  the 
incidents.  Bad  weather  was  the  second-most  frequent  reason;  it  accounted  for  one-third  of  the 
incidents. 

The  VTT  instructors  generally  indicated  that  training  missed  due  to  downtime  was  later 
made  up  so  that  no  training  was  lost  However,  since  students  had  to  wait  for  the  system  to  come 
back  up,  downtime  may  have  contributed  to  some  of  the  negative  attitudes  of  students  toward  the 
VTT  method  of  instruction. 

NET  SAVINGS  TO  THE  NAVY 

Net  savings  are  defined  as  the  difference  between  gross  savings  in  travel  and  per  diem  and 
the  cost  of  operating  the  VTT  system.  During  the  first  six  months,  VTT  generated  net  savings  of 
$68,721  to  the  Navy  as  a  whole.  However,  these  savings  should  not  be  used  to  project  the 
savings  that  would  be  achieved  by  any  future  system.  Both  the  benefits  and  costs  of  the 
future  system  are  likely  to  increase  as  the  system  expands  beyond  the  scale  of  the  present  system. 
In  the  short  run,  more  aggressive  scheduling  of  courses  with  high  throughput  should  yield 
additional  savings.  In  the  long  run,  changes  in  technology  should  lower  costs  and  improve  the 
quality  of  the  medium.  Further  cost-benefit  analyses  will  be  required  as  additional  data  on  the 
future  system  become  available. 

RECOMMENDATIONS 

The  cost-effectiveness  of  VTT  will  depend  critically  upon  the  extent  to  which  the  system  is 
used.  During  the  first  six  months,  fewer  than  half  of  all  classrooms  were  used  on  a  typical 
weekday.  To  improve  the  cost-effectiveness  of  VTT,  the  study  makes  the  following 
recommendations; 

•  Select  short  courses  with  high  throughput  per  convening  because  they  save  more  money 
per  week  than  other  types  of  courses. 

•  If  there  is  excess  capacity  in  the  training  system,  expand  the  “other  uses”  of  the  system 
because  they  generate  a  greater  than  proportionate  amount  of  gross  savings. 

•  Establish  an  R&D  program  to  (1)  develop  formal  courses  for  the  training  of  VTT  in- 
stmctors  and  (2)  redesign  course  materials  to  more  effectively  use  VTT  (e.g.,  graphics 
on  monitor). 

To  determine  the  maximum  number  of  sites  that  should  be  used  simultaneously  for  a  single 
course,  courses  should  be  convened  that  are  graded  and  that  use  three  or  more  remote  sites. 
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SECTION  1 


iNTRODUCTION 


The  Navy  is  exploring  alternative  methods  of  delivering  individual  training  in  a  cost- 
effective  manner  to  remote  locations.  Currently,  sailois  who  are  not  located  near  fleet  training 
centers  or  other  existing  “schcolhouses,”  must  either  travel  to  these  sites  or  have  the  training 
exported  to  them.  Sending  students  to  schoolhouses  is  very  expensive.  Alternatively,  when 
training  is  exported,  an  instructor  must  be  sent  to  the  remote  site  on  temporary  additional  duty 
(TAD).  Given  the  limited  number  of  instructors,  many  requests  for  exported  training  cannot  be 
filled.  In  FY  1989,  only  51.5  percent  of  the  quarterly  requests  to  COMTRALANT  for  exportable 
training  courses  were  approved. 

The  exportation  of  training  is  also  receiving  considerable  attention  in  the  planning  for 
strategic  homeporting.  The  advent  of  establishing  “strategic  homeports”  in  areas  that  are  even 
farther  from  existing  fleet  training  centers,  will  add  to  the  geographical  disbursement  of  potential 
students  and  could  cause  a  significant  drain  on  the  pool  of  available  instructors. 

This  study  evaluates  the  use  of  Video  Teletraining  (VTT)  to  deliver  instruction  to  students 
at  remote  locations.  VTT  uses  a  full  duple*  audio-visual  network  between  existing  schoolhouses 
and  remote  locations.  The  network  is  fully  interactive  in  that  participants  at  different  locations 
can  talk  to  and  see  one  another  in  real  time.  In  principle,  this  provides  the  “intimacy”  of  a  single 
integrated  classroom,  although  students  may  be  distributed  across  several  locations. 

The  initial  VTT  system  was  established  by  the  Navy  in  March  1989  with  joint  funding  from 
COMTRALANT,  CNET  and  QP-01.  A  multipoint  secure  system  was  designed  and  implemented 
with  sites  at  Charleston,  Dam  Neck,  Mayport,  and  Norfolk.  The  sites  are  interconnected  via  a 
commercial  Ku-band  satellite  through  a  CODEC  (coder/decoder),  a  microcomputer  to  compress/ 
decompress  the  video  signal,  and  a  video  branch  exchange  system  (VBX).  The  VBX  is  a  voice- 
activated  switch  tnat  selects  the  site  from  which  the  audio-visual  signal  will  originate.  The 
signals  are  encrypted  in  accordance  with  National  Security  Agency  (NSA)  standards  using 
KG-81s.  Training  up  to  the  level  of  secret  can  be  broadcast  on  the  VTT  network.  Figure  1  shows 
the  connectivity  among  the  system’s  elements. 

Cornpressing  the  video  signal  reduces  its  bandwidth.  This  results  in  a  significant  savings  in 
satellite  costs,  but  reduces  the  quality  of  the  signal.  However,  software  improvements  have 
permitted  reductions  in  bandwidth  during  the  evaluation  period  without  any  degradation  in  the 
quality  of  the  broadcast. 
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tgure  1 .  System  configuration  (SOURCE:  FCTCLAMT) 


Television  monitors,  video  cameras,  and  microphones  were  also  installed  in  each  classroom. 
A  copy  stand  was  installed  at  each  of  the  originating  sites  to  transmit  images  of  paper  copies  or 
small  equipment.  Figures  2  and  3  illustrate  the  initial  configurations  of  the  typical  VTf  class¬ 
rooms  at  the  originating  and  remote  sites,  respectively.  It  should  be  noted  that  pre-existing 
lighting  and  acoustical  conditions  were  used.  Due  to  budget  limitations,  no  special  effort  was 
made  to  design  a  classroom  to  accommodate  the  video  cud  audio  equipment. 


Figure  2.  Typical  originating  classroom  (Source:  FCTCIAITQ 

The  Center  for  Naval  Analyses  was  tasked  by  Commanaer,  Naval  Education  and  Training 
(CNET)  to  evaluate  this  system  during  its  first  six  months  of  operation  (i.e.,  March  1989  through 
September  1989).  Section  2  shows  the  extent  to  which  the  system  was  used  over  this  period. 
The  training  effectiveness  of  VTT  relative  to  traditional  method;,  of  instruction  is  examined  in 
sections  3  and  4.  Section  5  focuses  on  the  technical  problems  experienced  during  the  first 
six  months.  In  section  6,  the  net  savings  from  using  VTf  during  this  period  are  tabulated.  The 
main  conclusions  from  the  study  are  summarized  in  section  7. 
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SECTION  2 


SYSTEM  UTILIZATION 


This  section  shows  how  the  VTT  system  has  been  used  during  the  six-month  evaluation 
period.  Table  1  contains  a  list  of  the  first  12  courses  to  be  taught  via  this  medium.  These 
F-school-type  courses  tend  to  be  lecture -oriented  with  the  instructor  providing  hand-outs  and 
using  slides  or  transparencies.  The  key  exception  was  the  ,50-caiiber  Machine  Gun  Maintenance 
course  that  provided  some  hands-on  instruction  with  the  copy  stand. 


Table  1.  VTT  courses 


Course 

Length 
{in  days) 

Number  of 
convenings 

Originating 

site 

Number  of 
remote  sites 

Operation  Security  Planning 

2 

3 

Dam  Neck 

1 

Track  Supervisor,  Track  Force 

5 

4 

Dam  Neck 

2 

Coordinator 

Gun  Battery  Alignment 

5 

2 

Dam  Neck 

1 

Ammunition  Administration 

5 

3 

Dam  Neck 

1  to  3 

Advanced  Signalman 

10 

1 

Norfolk 

2 

Advanced  Communication  Procedures 

5 

3 

Norfolk 

1 

Command  Information  and  Retention 

20 

2 

Norfolk 

1 

Counselor/Coordinator  (CCCC) 

Theatre  Nuclear  Warfare 

2 

1 

Dam  Neck 

1 

SAS/EAP 

3 

2 

Dam  Nock 

1 

Soviet  Signaling  Procedures 

3 

2 

Norfolk 

1  to  2 

.50-caliber  Machine  Gun  Maintenance 

3 

1 

Dam  Neck 

2 

Nuclear  Weapons  Rad.  Con. 

3 

1 

Dam  Neck 

1 

The  courses  range  in  length  from  2  to  20  days;  the  typical  course  lasted  5  days.  During  the 
evaluation  period,  each  course  convened  from  1  to  4  times  for  a  total  of  25  convenings.  The 
originating  site  was  either  Dam  Neck  or  Norfolk,  and  the  remote  sites  were  typically  Charleston 
and/or  Mayport.  Only  one  convening  of  one  course  (i.e..  Ammunition  Administration)  used  all 
three  of  the  remote  sites.  For  this  convening,  Dam  Neck  was  the  originating  site,  and  Charleston, 
Mayport,  and  Norfolk  were  the  remote  sites. 

In  addition  to  using  the  VTT  system  to  conduct  Navy  course-training,  the  system  also 
supported  briefings  on  training  exercises,  civilian  training,  and  conferences  (see  table  2).  In 
contrast  to  the  VTT  courses,  a  typical  individual  used  die  system  lor  only  one  day.  An  event  like 
the  OCPM  conference  lasted  three  days,  but  llie  list  of  participants  was  different  on  each  day.  As 
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a  result,  this  conference  was  categorized  as  having  3  convenings  lasting  one  day  apiece.  Using 
this  definition,  there  were  8  convenings  involving  training  exercises,  4  convenings  of  civilian 
training,  and  4  convenings  of  conferencing.  For  these  other  uses,  all  four  of  the  sites  were 
employed  simultaneously  in  7  of  16  convenings  (44  percent).  For  course  training,  four  sites  were 
used  in  only  1  of  25  convenings  (4  percent). 


Table  2.  Other  uses  cf  VTT  system 


Usa  of  system 

Number  of 
convenings 

Number  of 
sites 

BFIT  exercise 

1 

5 

2 

CINTEX  exorcise 

1 

2 

3 

FLEETEX  exercise 

1 

1 

3 

TCCT  conference 

1 

1 

2 

Office  of  Civilian  Personnel 

Management  (OCPM)  Conference 

1 

3 

4 

OCPM  training 

1 

4 

4 

Table  3  indicates  how  intensively  the  network  of  six  classrooms  was  used  on  weekdays  over 
the  evaluation  period.1  On  die  average  weekday,  46.1  percent  of  the  classrooms  were  in  use. 
Utilization  rates  varied  substantially  across  sites,  ranging  from  28.4  percent  at  Dam  Neck  to 
72.7  percent  at  Mayport.  The  main  reason  for  this  variation  was  the  number  of  classrooms  at 
each  site.  The  Dam  Neck  and  Charleston  sites  have  much  lower  utilization  rates  than  Norfolk 
and  Mayport  but  twice  as  many  VTT  classrooms.  Thus,  the  second  classroom  at  a  given  site  was 
typically  not  used.  This  problem  has  recently  been  addressed  by  eliminating  the  second  class¬ 
rooms  at  Dam  Neck  and  Charleston  and  establishing  a  new  site  at  Newport.  Because  these 
changes  took  place  after  the  first  six  months,  one  should  expect  higher  system  utilization  during 
die  next  year. 

Table  4  provides  data  on  attendance  for  each  type  of  use.  A  total  of  705  students  were 
trained  in  25  convenings  of  die  12  VTT  courses  fisted  in  table  l.2  In  contrast,  the  total  atten¬ 
dance  from  other  uses  of  die  system  was  higher  (737  persons),  but  diis  was  achieved  using  only 
16  convenings.  The  other  military  uses  of  the  system  consisted  primarily  of  Navy  training 
exercises,  and  the  civilian  uses  of  die  system  consisted  primarily  of  OCPM  training.  The  civilian 
uses  had  die  highest  attendance  per  convening  with  an  average  of  55.1  persons.  This  was 
accomplished  by  using  all  four  sites  at  each  of  dieir  convenings.  The  other  military  uses  hail  the 
next  highest  attendance  |>er  convening,  with  an  average  of  39  persons.  Although  they  used  an 


! .  Legal  holidays  aie  excluded  from  ilk-  analysis. 

2.  Appendix  A  contains  a  detailed  breakdown  ol  attendance,  by  convening  and  site. 


average  of  only  2.3  sites  per  convening,  the  other  military  uses  had  the  highest  attendance  per 
site,  with  an  average  of  16.7  persons.  The  VTT  courses  had  the  lowest  attendance  per  convening, 
with  an  average  of  28.2  students.  The  relatively  low  attendance  for  the  average  VTT  course  can 
be  attributed  to  the  fact  that  the  courses  used  the  fewest  number  of  sites  per  convening  (i.e., 
2.4  sites)  and  had  the  lowest  attendance  per  site  (i.e.,  1 1 .8  students). 


Table  3.  Classroom  utilization  rates  by  site 


Proportion  of  classrooms 
Number  of  in  use  per  weekday 

Site  classrooms  (percent) 


Charleston 

2 

41.3 

Dam  Neck 

2 

28.4 

Mayport 

1 

72.7 

Norfolk 

1 

64.4 

All  sites 

6 

46.1 

Table  4.  Attendance  by  type  of  use 

VTT 

courses 

Other  us 

Military 

as 

Civilian 

Number  of  personnel 

705.0 

351.0 

386.0 

(oxcluding  instructors) 
Number  of  convenings 

25.0 

9.0 

7.0 

Attendance  per  convening 

28.2 

39.0 

55.1 

Sites  per  convening 

2.4 

2.3 

4.0 

Attendance  par  site 

11.8 

16.7 

13.8 

The  next  two  sections  of  this  memorandum  address  the  effectiveness  of  using  VTT  for 
Navy  courses.1  Training  effectiveness  is  examined  in  section  III  using  course  grades  as  the 
measure  of  effectiveness.  Section  4  reports  on  a  survey  of  VTT  students  tint  was  conducted  to 
identify  specific  training  deficiencies. 


1.  The  Naval  Training  System  Center  (NTSC)  was  tasked  by  CNET  to  provide  a  separate  analysis  of 
VTT's  effectiveness  in  its  other  military  and  civilian  uses. 
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SECTION  3 


EFFECT  OF  VTT  ON  COURSE  GRADES 


This  section  employs  a  regression  model  to  evaluate  the  relative  effectiveness  of  using  VTT  in 
conducting  training  for  the  Navy.  Table  5  defines  each  of  the  variables  in  this  model.  Final  course 
grades  are  the  dependent  variable  because  they  provide  a  measure  of  learning  effectiveness.  The 
key  independent  variable,  REMOTE,  indicates  whether  a  given  student  was  at  a  remote  or  originat¬ 
ing  site.  Thus,  the  grades  of  students  at  the  remote  sites  are  compared  to  those  at  the  originating 
site  to  determine  how  effective  the  VTT  method  of  instruction  was,  relative  to  the  traditional 
method  of  instruction.  Students  at  the  remote  sites  serve  as  the  test  group  in  the  evaluation  because 
they  could  see,  hear,  and  speak  with  the  instructor  exclusively  through  the  VTT  technology.  In 
contrast,  students  in  the  same  room  as  the  instructor  (i.e.,  the  originating  site)  serve  as  the  control 
group  because  they  interacted  with  the  instructor  without  using  the  VTT  technology. 


Tabla  5.  Variables  in  regression  model 


Variable 

Definition 

Source  of  data 

GRADE 

Final  course  grade  on  scale  of  100  points 

Instructor  grade  sheets 

REMOTE 

1 :  student  at  remote  site 

0:  student  at  originating  site 

VTT  student  background  questionnaires 

NSITES 

1:  three  sites 

0:  two  sites 

VTT  course  schedules 

SIZE 

Number  of  students  at  all  sites 

VTT  status  reports 

AFQT 

Percentile  score  on  Armed  Forces 
Qualification  Test 

Enlisted  Master  Record 

YOS 

Years  of  service  as  of  January  1989 

Enlisted  Master  Record 

COURSE1 

1 :  T rack  Supervisor  cou  rse 

0:  otherwise 

VTT  student  background  questionnaires 

COURSE2 

1 :  Advanced  Signalman  course 

0:  otherwise 

VTT  student  background  questionnaires 

COURSE3 

1 :  Advanced  Communication  Procedures 

course 

0:  otherwise 

VTT  student  background  questionnaires 

COURSE4 

1 :  CCCC  course 

0:  otherwise 

VTT  student  background  questionnaires 

COURSE5 

1 :  Soviet  Signaling  course 

0:  otherwise 

VTT  student  background  questionnaires 
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Although  students  at  the  originating  site  are  the  best  available  control  group,  they  are  not  a 
perfect  control  group  since  some  of  their  learning  did  rely  on  the  VTT  technology.  For  example, 
when  students  were  speaking  at  one  of  the  remote  sites,  students  at  the  originating  site  had  to  rely 
on  the  VTT  technology  hear  and  see  ihem.  As  the  number  of  sites  increases,  one  would  expect 
that  the  proportion  of  i  ae  spent  on  listening  and  hearing  students  at  the  other  sites  would 
increase.  The  model  addresses  this  issue  by  controlling  for  the  number  of  sites,  NSITES,  in  the 
regression  equation.  This  variable  would  also  control  for  difficulties  that  the  instructor  might 
have  in  teaching  to  a  larger  number  of  sites  and  to  a  larger  number  of  students.  The  effect  of 
class  size  can  be  quantified  separately  by  including  the  total  number  of  students,  SIZE,  in  the 
equation  and  observing  its  effect  on  the  coefficient  of  NSITES. 

Comparing  grades  of  students  at  the  originating  site  with  those  of  students  at  remote  sites  will 
not  show  the  relative  effectiveness  of  using  the  copy  stand  in  communicating  visual  information. 
When  an  instructor  uses  a  copy  stand,  students  at  both  types  of  sites  observe  the  material  via  their 
television  monitors.  However,  when  the  instructor  focuses  one  of  the  cameras  on  a  blackboard  or 
slide  screen,  students  at  the  originating  site  observe  the  material  without  relying  on  the  VTT  tech¬ 
nology  whereas  students  at  the  remote  sites  again  rely  on  the  technology  via  their  monitor. 

The  VTT  system  was  designed  to  provide  actual  training  rather  than  to  serve  as  a  laboratory 
for  manipulating  variables  such  as  the  configuration  of  the  room  or  type  of  equipment  There¬ 
fore,  the  variables  used  in  the  regression  model  were  limited  to  those  variables  that  differed 
across  classrooms  or  individuals. 

The  regression  model  controls  for  differences  in  student  mental  aptitude  and  years  of 
service  by  including  the  variables  AFQT  and  YOS.  In  addition,  differences  in  grading  across 
courses  are  captured  by  the  course  dummy  variables,  COURSE!  through  COURSE5.  These 
five  variables  respectively  correspond  to  Track  Supervisor  Track  Force  Coordinator,  Advanced 
Signalman,  Advanced  Communication  Procedures,  CCCC,  and  Soviet  Signaling.  The  coefficient 
of  a  given  dummy  variable  is  an  estimate  of  the  difference  in  grades  between  that  particular 
course  and  the  Gun  Battery  Alignment  course. 

The  sample  consists  of  356  students  from  a  total  of  13  course  convenings.  The  other 
12  convenings  cannot  be  used  in  the  regression  analysis  because  cither  grades  were  not  given  or 
no  students  took  the  course  at  the  originating  site.  Furthermore,  only  enlisted  personnel  are 
included  in  the  sample  because  of  the  need  to  control  for  the  AFQT  score.5  Half  of  the  students 
in  the  sample  received  the  training  at  the  originating  site,  and  the  other  half  received  the  training 
at  one  of  the  remote  sites. 

Table  6  presents  lire  results  from  tire  regression  analysis.  Controlling  for  each  of  the 
variables  discussed  above,  grades  at  the  remote  sites  were  2.4  points  lower  than  grades  at  the 


1.  Oi  lire  total  population  in  lire  first  25  VTT  convenings,  81  4  percent  were  enlisted  and  18.5  percent  were 
officers. 
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originating  site  on  average.  The  2.4-poirit  difference  is  statistically  significant  at  the  1  percent 
level  and  corresponds  to  one-third  of  a  standard  deviation  in  the  grade  measure. 


Table  6.  Regression  resuits  (dependent  variable:  GRADE) 


Coefficient  (t-statistic) 


Independent 

variable 

Mean 

(standard  deviation) 

Model  without 

SIZE  variable 

Model  with 
SIZE  variable 

Intercept 

80.3s 

80.9® 

(43.9) 

(36.7) 

REMOTE 

0.50 

-2.4s 

-2.4a 

(0.50) 

(-3-8) 

(-3.8) 

NSITES 

0.44 

-2.2b 

-1.8 

(0.50) 

(-1.9) 

(-1-2) 

SIZE 

31.2 

-.03 

(7.9) 

(-.5) 

AFQT 

62.2 

.15a 

.15-’ 

(18.1) 

(7.7) 

(7-7) 

YOS 

5.9 

,34a 

.34® 

(4.4) 

(3-5) 

(3.4) 

COURSE1 

.33 

-2.0 

-1.8 

(.47) 

(-1.2) 

HI) 

COURSE2 

.06 

-3.1 

-3.2 

(.24) 

(-1-5) 

(-1.6) 

C.OURSE3 

.17 

-3.1b 

-2.8b 

(.37) 

(-2.3) 

(-1.9) 

COURSE4 

.19 

1 .7 

-1.1 

(.39) 

(-1.2, 

(-6) 

COURSE5 

.17 

-3. 2s 

-3.0b 

(37) 

(-2.1) 

(1.9) 

R-squara 

.33 

.33 

F -statistic 

1 8.5a 

1 6. 7a 

Sample  size 

356.0 

356.0 

NOTE  Mean  and  standard  deviation  ot  GRADE  are  E7  2  and  7  2,  respectively 

a  Statistically  significant  at  the  1  porcenl  love' 
b  Statistically  significant  at  the  5  percent  level 


When  the  total  number  of  sites  was  increased  from  two  to  three,  the  average  grade  dropped 
2.2  points  at  both  originating  and  remote,  sites.  In  the  first  six  months,  there  have  not  been  any 
graded  VTT  courses  using  three  or  more  remote  sites.  Whether  grades  would  remain  the  same  or 
further  deteriorate  with  an  increase  from  two  remote  sites  to  three  or  more  remote  sites,  remains 
an  important  question  for  further  research. 

When  the  number  of  students  is  included  as  an  explanatory  variable,  it  reduces  the  coeffi¬ 
cient  on  the  number  of  sites  variable  from  2.2  to  1.8.  This  indicates  that  the  increase  in  class  size 
explaias  only  18  percent  of  the  reduction  in  grades  associated  with  an  increase  in  the  number  of 
sites.  Thus  class  size  only  partially  explains  the  negative  effect  on  grades  of  increasing  the 
number  of  sites. 

The  effect  of  individual-specific  variables  on  course  grades  is  in  the  anticipated  direction. 
An  increase  of  one  standard  deviation  from  the  mean  AFQT  score  (i.e,,  from  the  62nd  to 
80th  percentile),  improved  the  course  grade  by  2.7  points  on  average.  Likewise,  an  increase  of 
one  standard  deviation  from  the  mean  Navy  experience  (i.e.,  from  6  to  10  years  of  service),  was 
associated  with  a  1.5-point  increase  in  the  course  grade  on  average. 

The  regression  results  in  table  6  are  robust.  Substituting  convening-specific  dummy 
variables  for  the  course-specific  dummy  variables  leaves  the  results  intact.1  Interacting  the 
number  of  sites  with  whether  the  student  is  at  the  originating  or  remote  sites,  confirms  that 
students  at  both  the  originating  and  remote  sites  experienced  on  average  the  same  2.2-point 
reduction  when  the  total  number  of  sites  was  increased  from  two  to  three.2 

From  a  policy  standpoint,  the  question  arises  whethes  the  difference  in  grades  between 
originating  and  remote  sites  is  practically  significant.  The  Navy’s  VTT  Steering  Committee  has 
chosen  differences  in  failure  rates  as  the  relevant  criterion  for  measuring  “practical  signifi¬ 
cance.”  3  Students  who  received  final  course  grades  below  70  were  classified  as  failures  based 
on  the  school’s  definition  of  failure  in  a  typical  course. 

At  (he  originating  site,  only  3  of  178  students  (1.7  percent)  received  grades  below  70.  Al 
the  remote  sites,  only  4  of  178  (2.2  percent)  received  grades  below  70.  Therefore,  the  difference 
in  grades  during  the  evaluation  period  is  not  “practically  significant"  iiecause  the  difference  in 
failure  rates  between  originating  and  remote  sites  is  small  and  statistically  insignificant  at  the 
5-percent  level. 


1.  'Ihc  NSITES  variable  must  be  dropped  when  using  this  specification  of  the  model  because  the  number 
of  sites  is  a  linear  combination  of  the  convening  specific  dummy  variables. 

2.  Results  from  these  alternative  specifications  of  the  model  are  documented  in  apixmdix  B 

3.  The  Y"1T  Steering  Committee  consists  of  representatives  from  a  wide  variety  of  Navy  organizations 
including  CNT.T  and  the  functional  training  commands. 


Failure  rates  were  not  affected  by  differences  in  grades  between  sites  because  the 
mean  grades  at  both  types  of  sites  were  much  higher  than  the  minimum  passing  score  of  70. 
Controlling  for  the  other  factors  affecting  grades,  the  mean  student  grades  were  88.4  at  the 
originating  site  and  86.0  at  the  typical  remote  site.1  Figure  4  shows  the  cumulative  distributions 
of  grades  for  students  at  the  originating  and  remote  sites  based  on  the  grade  data  collected  during 
the  evaluation  period.  Note  that  most  of  the  grade  distribution  lies  above  the  minimum  passing 
score  of  70.  If  the  shapes  of  these  distributions  and  the  average  difference  between  sites  are 
assumed  to  be  the  same  for  courses  with  different  mean  grades,  the  effects  of  using  VTT  in  more 
difficult  courses  can  be  estimated. 


Figure  4.  Course  grade  distributions  at  originating  and  remote  VTT  sites 


Figure  5  provides  estimates  of  expected  failure  rates  at  originating  and  remote  sites  for 
different  mean  grades  at  the  typical  remote  site.  Note  that  the  corresponding  mean  grade  at  the 
originating  site  is  2.4  points  higher  than  the  hypothetical  mean  grade  at  the  remote  site.  During 
the  evaluation  period,  the  mean  grade  at  the  remote  site  was  86.0.  The  difference  in  failure  rates 
between  the  originating  and  remote  sites  at  this  point  is  one-half  of  a  percentage  point 

As  more  difficult  courses  are  selected  (i.e.,  courses  with  mean  grades  below  86.0),  the 
difference  in  failure  rates  will  increase.  If  the  mean  grade  falls  to  79.0  (i.e.,  approximately 
one  standard  deviation  from  the  sample  mean),  the  failure  rates  at  the  originating  and  remote 
sites  increase  to  5.1  percent  and  10.7  percent,  respectively.  The  difference  in  failure  rates  is  then 


1.  The  typical  remote  site  is  associated  with  a  convening  consisting  of  one  originating  site  and  1.4  remote 
sites  (i.e.,  the  mean  number  of  sites  during  the  evaluation  period). 
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5.6  percentage  points,  which  is  statistically  significant  at  the  5-percent  level.  On  the  other  hand, 
if  less  difficult  courses  are  selected  (i.e.,  courses  with  mean  grades  above  86.0),  there  is  virtually 
no  difference  in  failure  rates  between  sites. 


Mguie  5.  Predicted  failure  rates  at  originating  and  remote  sites 
failure,  grade  below  70) 


These  results  suggest  that  the  level  of  difficult)  of  a  course  is  an  important  criteria  in 
selecting  those  courses  best  suited  for  the  V  IT  method  of  instruction.  It  should  be  noted, 
however,  that  the  difference  hi  failure  rates  would  decrease  if  the  grades  at  remote  sites  improve 
in  the  future  relative  to  the  grades  at  originating  sites. 


SECTION  4 


STUDENT  SURVEY 


This  section  uses  student  survey  data  to  identify  specific  areas  for  improving  the  effective¬ 
ness  of  'VTT.  On  the  first  day  of  each  convening,  background  questionnaires  were  administered 
to  students  at  each  of  the  sites.  Student  responses  were  used  to  identify  the  sailors  who  were 
trained,  the  site  where  they  received  the  training,  the  location  of  their  duty  station,  and  their 
travel  and  berthing  arrangements.  On  the  last  day  of  each  convening,  course  questionnaires  were 
administered  to  students  at  each  of  the  sites.  Students  were  asked  to  evaluate  the  instructor, 
audio-visual  aids,  tests,  homework,  instructor-student  interaction,  and  remedial  instruction.  In 
addition,  they  were  asked  to  indicate  their  preferences  as  to  method  of  instruction.1 

The  questionnaires  were  typically  administered  by  the  facilitators  at  each  site.  Copies  of  the 
forms  were  mailed  to  the  the  point-of-contact  at  the  Fleet  Combat  Training  Center,  Atlantic 
(FCTCLANT)  and  then  forwarded  to  CNA  for  analysis.  For  each  student,  data  from  the 
two  questionnaires  were  entered  into  a  VTT  data  base.  Of  the  students  who  took  a  VTT  i  rse, 
95  percent  of  them  returned  their  course  questionnaires.2  In  cases  where  students  did  not 
complete  a  background  questionnaire  but  did  complete  a  course  questionnaire,  most  of  die 
information  required  for  the  background  questionnaire  was  obtained  from  other  sources. 

Results  from  the  first  section  of  the  course  questionnaire  are  contained  in  tables  7  through 
10.  For  each  item,  students  were  asked  to  choose  a  number  from  a  scale  of  1  to  5,  in  which  1  is 
unsatisfactory  and  5  is  outstanding.  In  the  analysis,  the  mean  response  was  calculated  for  each 
item  by  type  of  site  (i.c.,  remote  versus  originadng).  For  a  given  item,  the  difference  in  the  mean 
response  between  remote  and  originating  sites  is  then  computed  and  a  t-test  is  performed  to 
determine  whether  the  difference  is  statistically  significant.  The  sample  sizes  for  each  item  by 
type  of  site  and  the  actual  t-statistics  are  contained  in  appendix  D. 

Results  from  die  student  evaluation  of  die  instructor  are  shown  in  table  7.  On  each  of  the 
items,  sludents  at  the  remote  sites  rated  the  instructor  significantly  lower  than  students  at  the 
originating  site.  The  largest  difference  between  sites  is  on  die  item  “availability  of  the  instructor 
for  individual  assistance  outside  of  class."  Although  instructors  were  advised  to  set  aside  time 
after  class  to  speak  with  students,  the  instructors  indicated  diat  students  at  remote  sites  were  less 
likely  to  take  advantage  of  these  opportunities  dian  students  at,  the  originating  site. 


1 .  Appendix  C  contains  copies  of  the  .student  questionnaires. 

7.  Three  convenings  accounted  for  all  of  the  missing  data:  the  1 1th  convening  (Theatre  Nuclear  Warfare), 
the  17th  convening  (SAS/EAP),  anu  die  24th  convening  (.50-caliber  Machine  Gun  Maintenance). 
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TabU  7.  Student  evaluation  of  instructor 


i 


4 


Mean  response  (scale  1-5) 


Statement 

Remote 

Originating 

Difference 

Instructor  prepared  for  class. 

4.6 

4.9 

-.3 

Instructor  presented  lessons  clearly. 

4.1 

4.7 

-.6 

Instructor  answered  student  questions. 

4.5 

4.9 

-.4 

Instructor  encouraged  class  participation. 

4.2 

4.6 

-.4 

Instructor  was  available  for  individual 
assistance  outside  of  class. 

3.1 

4.7 

-1.6 

Instructor  treated  students  fairly. 

4.6 

4.9 

-.3 

NOTE:  For  each  item,  the  difference  is  statistically  significant  ct  the  1-percent  level. 


Table  8.  Student  evaluation  of  audic-visual  aids 


Mean  response  (scale  1  -5) 


Statement 

Remote 

Originating 

Difference 

Video  screen  was  large  enough  to  be  seen. 

4.0 

4.4 

~.4a 

Video  screen  was  close  enough  to  be  seen. 

4.3 

4.5 

-,2s 

Image  on  video  screen  was  clear. 

3.3 

4.1 

-,8a 

Audio  transmission  was  loud  enough  to 

3.6 

4.3 

-,5a 

hear  instructor’s  voice. 

Audio  transmission  was  clear  enough  to 

3.5 

4.3 

-.0a 

hear  what  instructor  said. 

Graphics/slides/transparcncies  on  TV 

2.7 

4.1 

-1.4a 

were  readable. 

Television  was  in  working  order. 

4.3 

4.3 

,0b 

Your  microphone  was  in  working  order. 

4.3 

4.5 

_  pb 

a.  Difference  is  statistically  significant  at  the  1  percent  level. 

5.  Difference  is  nor  statistics  "y  significant  at  the  5-  percent  level. 
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Table  9.  Student  evaluation  of  tests  and  homework 


Mean  response  (scale  1-5) 


Statement 

Remote 

Originating 

Difference 

Test  questions  were  clearly  written. 

4.4 

4.6 

-.2 

Test  questions  were  directly  related  to  course. 

4.7 

4,9 

-.2 

Test  answers  were  graded  fairly. 

4.7 

4.9 

-.2 

Homework  assignments  were  understandable. 

4.5 

4.7 

-.2 

Homework  assignments  were  directly  related 

4.7 

4.9 

-.2 

to  course. 


NOTE:  For  each  item,  the  difference  is  statistically  significant  at  the  1-percent  level. 


Table  10.  Student  o'-erall  evaluation  of  instructor  and  course 


Mean  response  (scale  1-5) 


Statement 

Remote 

Originating 

Difference 

Comparison  of  this  instructor  to  other 

3.9 

4.6 

-.7 

Navy  instructors  that  have  taught  you  in  the  oas> 

Comparison  of  this  course  to  other  Navy  courses 

3.6 

4.4 

-.8 

that  you  have  taken  in  the  past 


NOTE:  For  each  item,  the  difference  is  statistically  significant  at  the  1 -percent  level. 


Table  8  reports  student  mean  responses  to  the  audio  visual  aids  used  in  ihe  course.  In 
contrast  to  the  previous  table  it  should  be  noted  that  students  at  the  originating  site  spend  less 
time  using  these  aids  and  therefore  have  less  information  with  which  to  judge  their  effectiveness. 
In  particular,  students  were  asked  about  then  ability  to  hear  the  instructor  through  the  audio 
transmission.  For  students  at  the  originating  site,  this  item  was  not  strictly  applicable.  However, 
they  migln  have  inferred  the  general  quality  of  the  audio  transmission  from  their  ability  to  hear 
(he  students  at  the  remote  sites. 

Students  at  the  remote  sites  must  rely  on  VTT’s  audio-visual  aids  for  most  of  their  instruc¬ 
tion.  With  the  exception  of  the  last  two  items  (i.e.,  television  ana  microphone  in  working  order), 
these  students  rated  audit -visual  aids  significantly  lowc  than  students  at  live  originating  site. 


Furthermore,  students  at  the  remote  sites  tended  to  rate  the  audio-visual  aids  lower  than  other 
items  in  the  course  evaluation.  In  table  7,  the  average  response  at  the  remote  sites  was  less 
than  4  on  only  one  of  six  items  concerning  the  instructor  (i.e.,  instructor  availability  for 
individual  assistance  outside  of  class).  In  table  8,  the  average  response  was  less  than  4  on  four  of 
eight  items  concerning  the  audio-visual  aids.  The  two  lowest-rated  items  in  the  evaluation  are 
related  to  the  quality  of  the  video  (i.e.,  readability  of  graphics/slides/transparencies  on  TV  and 
clarity  of  image  on  video  screen).  The  other  two  items  to  receive  ratings  of  less  than  4  are 
related  to  the  quality  of  the  audio  (i.e.,  loudness  and  clarity  of  audio  transmission). 

Results  from  the  students’  evaluation  of  tests  and  homework  are  shown  in  table  9.  Al¬ 
though  students  rated  each  of  these  five  items  significantly  lower  at  the  remote  sites  than  the 
originating  site,  the  magnitude  of  the  difference  between  sites  is  very  small  (i.e.,  two-tenths  of 
one  point  on  a  scale  of 1  to  5).  Indeed,  the  difference  between  sites  is  smaller  on  items  concern¬ 
ing  tests  and  homework  than  on  items  concerning  the  instructor  or  audio-visual  aids.  In  addition, 
none  of  the  five  test/homework  items  is  rated  below  4. 

Based  on  the  results  from  tables  7  through  9,  the  main  problem  areas  were  the  quality  of  the 
video  transmission,  the  use  of  the  instructor  for  individual  assistance  outside  of  class,  and  the 
quality  of  the  audio  transmission.  Students’  dissatisfaction  with  these  items  may  have  spilled 
over  to  other  items  and  led  to  general  dissatisfaction  with  the  course  and  instructor.  Table  10 
demonstrates  that  students  at  the  remote  sites  rated  both  the  instructor  and  the  course  sig¬ 
nificantly  lower  than  students  at  the  originating  site. 

Not  surprisingly,  table  11  shows  that  students  generally  preferred  the  traditional  method  of 
instruction  to  VTT.  The  proportion  of  students  favoring  the  traditional  method  was  significantly 
lower  at  the  remote  sites  (64.2  percent)  than  at  the  originating  site  (78.2  percent).1 


Table  11.  Student  preferences  on  method  of  instruction 
(percent) 


Question:  Which  method  of  instruction  would  you  have 

preferred  for  this  course? 

Method 

Remote 

Originating 

Traditional 

64.2 

78.2 

Indifferent 

27.2 

19.5 

VTT 

8.6 

2.3 

100.0 

100.0 

1.  In  appendix  D,  tables  D-5  and  D-6  provide  l-slatistics  and  sample  sizes,  respectively,  for  statements  in 
tables  1 1  through  16. 
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Tab!«  12.  Student  interaction  with  instructor  (percent) 


Question:  Did  you  talk  to  the  instructor  or  ask  any  questions 
during  the  regular  hours  of  this  course? 


Response 

Remote 

Originating 

Yes 

75.6 

92.4 

No 

24.4 

7.6 

100.0 

100.0 

Table  13.  Student  evaluation  of  opportunities  for  interaction 
(percent) 


Question:  How  did  VTT  affect  your  opportunities  tn  talk  to  the 
instructor  or  ask  questions,  as  comnared  to  traditional  methods 
cf  instruction? 


Response 

Remote 

Originating 

More  opportunities 

1.8 

5.6 

No  e fleet  on  opportunities 

43.2 

79.3 

Fewer  opportunities: 

55.0 

'  5.1 

1 00.0 

100.0 

Tabl«  14,  Student  use  oi  remedial  instruction  (percent) 


Question:  D'd  you  attend  any  remedial  instruction  pehoos? 


Response 

Remote 

Originating 

Yos 

13.4 

21  .C 

No 

86.6 

78.2 

100.0 


100.0 


Table  15.  Student  evaluation  of  opportunities  for  remedial 
instruction  (percent) 


Question:  Were  there  adequate  opportunities  for  remedial 
instruction  outside  of  the  regular  class  hours  of  this  course? 


Response 

Remote 

Originating 

Yes 

41.6 

73.2 

No 

23.4 

2.3 

Remedial  instruction  was  not 

35.0 

24.5 

necessary  for  this  course 

100.0 

100.0 

Table  16.  Student  evaluation  of  participation  at  other  sites 
(percent) 


Question:  How  did  the  participation  of  students  at  other  site(s) 
affect  your  learning  during  this  course? 


Response 

Remote 

Originating 

Improved  learning 

12.2 

16.2 

No  effect  on  learning 

69.6 

66.3 

Reduced  learning 

18.2 

17.5 

100  0 

100.0 

The  advantage  of  using  two-way  video/audio  as  opposed  to  one-way  vidco/audio  is  that  the 
fonner  permits  students  at  a  remote  site  to  interact  with  the  instructor  and  students  at  other  sites. 
Under  the  traditional  method  of  instruction,  such  interaction  could  easily  be  facilitated  by  the  fact 
that  the  instructor  is  in  the  same  classroom  with  all  of  the  studen  Although  two-way  video/ 
audio  technically  permits  interaction  between  sites,  whether  people  will  fully  use  this  capability 
is  an  important  issue.  Tables  12  through  16  show  the  extent  to  which  instructors  and  students 
interacted  under  VTT  relative  to  traditional  metliods  of  instruction.  Students  at  tlx:  originating  site 
interacted  with  the  instructor  via  the  traditional  method  since  the  instructor  was  in  die  same  room 
Students  at  tlx:  remote  sites  interacted  with  Ihe  instructor  only  by  means  of  the  VTT  system. 
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Table  12  reveals  that  students  at  the  remote  sites  were  less  likely  to  interact  with  the 
instructor  than  students  at  the  originating  site.  The  proportion  of  students  indicating  that  they  did 
not  talk  to  the  instructor  or  ask  any  questions  during  the  course’s  regular  hours  was  24.4  percent 
at  the  remote  sites  but  only  7.6  percent  at  the  originating  site,  for  a  difference  of  16.8  percentage 
points. 

Students  were  asked  to  assess  how  VTT  affected  their  opportunities  to  interact  with  the 
instructor  (see  table  13).  At  the  originating  site,  a  majority  of  students  (79.3  percent)  believed 
that  VTT  had  no  effect  on  their  opportunities  to  talk  to  the  instructor  or  ask  questions.  At  the 
remote  sites,  a  majority  of  students  (55.0  percent)  believed  that  VTT  reduced  their  opportunities 
to  interact  with  the  instructor.  In  sharp  contrast,  only  15.1  percent  of  the  students  at  the  originat¬ 
ing  site  believed  that  VTT  reduced  their  opportunities  for  interaction,  for  a  difference  of 
39.9  percentage  points  between  sites. 

If  students  at  remote  sites  are  more  reluctant  than  students  at  the  originating  site  to  interact 
with  the  instructor  during  regular  class  hours,  arc  they  more  likely  to  interact  with  him  outside  of 
regular  class  hours  when  fewer  students  are  using  the  system?  Table  14  shows  that  this  was  not 
the  case  in  the  evaluation  period.  In  fact,  students  at  remote  sites  (13.4  percent)  were  even  less 
likely  to  attend  remedial  instruction  periods  than  students  at  the  originating  site  (21.8  percent). 
Table  15  indicates  that  a  majority  of  students  at  the  originating  site  (73.2  percent)  felt  that  there 
were  adequate  opportunities  for  remedial  instruction  whereas  a  minority  of  students  at  the  remote 
sites  (41.6  percent)  expressed  that  view. 

These  results  suggest  that  the  interaction  between  the  instructor  and  students  at  remote  sites 
needs  to  be  improved.  It  should  be  noted  that  these  courses  were  not  specifically  designed  for 
the  VTT  system.  Moreover,  there  are  currently  no  formal  courses  to  train  instructors  in  using 
VTT  and  no  procedures  for  selecting  instructors  best  suited  for  this  medium.  The  Naval  Training 
System  Center  (NTSC)  has  completed  a  literature  review  that  provides  valuable  information  the 
Navy  could  use  in  redesigning  courses  and  retraining  instructor  for  VTT. 1 

Students  were  asked  about  how  the  participation  of  other  students  at  oilier  sites  affected 
their  learning  (sec  table  16).  Although  most  students  at  both  types  of  sites  believed  that  it  had  no 
effect,  a  greater  proportion  of  students  at  the  originating  site  believed  that  this  participation 
improved  their  learning. 

In  the  previous  section,  it  was  shown  dial  increasing  the  number  of  sites  reduced  course 
grades.  However,  the  survey  data  indicates  that  there  were  no  differences  in  attitudes  between 
students  who  were  trained  in  convenings  with  two  versus  three  sites. 


1.  See  Naval  Training  Systems  Center,  Video  Teletraining  and  Video  Teleconferencing:  A  Review  of  the 
Literature ,  Technical  Report  K9-C8901X,  Draft  Version,  October  1989. 
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SECTION  5 


DOWNTIME 


When  the  transmission  is  degraded  to  a  large  degree  or  lost  altogether,  a  VTT  course  must 
stop  and  the  time  that  students  must  wait  for  the  course  to  restart  is  defined  as  downtime.  In  this 
evaluation,  if  the  incident  occurred  near  the  end  of  the  class-day  and  the  class  reconvened  earlier 
on  the  next  day,  the  system  is  classified  as  down  but  the  amount  of  downtime  is  recorded  as  zero. 

The  VTT  instructors  generally  indicated  that  the  training  that  was  missed  due  to  downtime 
was  later  made  up  so  that  no  training  was  lost.  However,  because  downtime  disrupted  training 
and  caused  students  to  wait  for  the  system  to  come  back  up,  it  could  have  affected  their  attitudes 
towards  the  course  and  the  VTT  system. 

When  the  system  was  down,  the  VTT  project  manager  at  the  FCTCLANT  was  responsible 
for  coordinating  the  activities  required  to  bring  the  system  back  up.  hi  discharging  these  duties,  a 
record  was  maintained  of  such  events  in  a  weekly  status  report.  For  each  incident,  the  amount  of 
downtime  was  recorded  as  well  as  the  suspected  reason  for  the  failure  of  the  system.  These  data 
are  summarized  in  tables  17  and  18. 


Table  17.  Reported  downtime  per  class-day 


Month 

Downtime3 

(minutes) 

Class -days 

Downtime 
per  class-day 

March 

250 

17 

14.7 

April 

45 

15 

3.0 

May 

too 

25 

4.0 

June 

190 

18 

10.6 

July 

214 

25 

8.6 

August 

186 

26 

7.2 

September 

0 

5 

0.0 

Total 

985 

131 

7.5 

a  Downtime  counts  twice  if  it  affected  two  courses  simultaneously 


Table  17  shows  that  a  total  of  985  minutes  of  downtime  (i.e.,  16.4  hours)  occurred  during  a 
lota!  of  131  class-days,  for  an  average  of  7.5  minutes  per  class-day.  File  amount  of  downtime 
varied  substantially  by  month.  The  greatest  amount  of  downtime  per  day  (14.7  minutes)  oc¬ 
curred  during  die  first  month  of  operation.  A  NASA  launch  of  the  space  shuttle  accounted  for 
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almost  half  of  the  March  downtime  (120  of  250  minutes).  At  that  time,  the  VTT  system  was  not 
considered  a  “regular”  user  of  the  satellite.  Consequently,  the  VTT  system  was  “bumped”  off  the 
satellite  that  it  was  using  and  was  forced  to  find  an  alternative  satellite  for  its  transmission.  This 
problem  was  later  remedied  and  subsequent  launches  of  the  space  shuttle  have  not  accounted  for 
any  downtime. 


Table  18.  Reasons  for  downtime 


Causa 

Number 
of  times 

Minutes 

Percentage  of 
total  downtime 

Equipment  problems 

15 

415 

4C.1 

Bad  weather 

10 

235 

31.7 

NASA  launch 

1 

120 

13.3 

Power  outage 

4 

80 

8.9 

Total 

30 

900 

100.0 

a.  Downtime  counts  once 

if  it  affected  two  courses 

simultaneously. 

The  general  reasons  for  downtime  are  shown  in  table  18.1  Equipment  problems  were  the 
most  frequent  cause;  they  accounted  for  half  of  the  incidents  (15  of  30)  and  46.1  percent  of  the 
actual  downtime.  Bad  weather  was  the  second-most  frequent  reason  for  the  system  going  down. 
It  accounted  for  one-third  of  the  incidents  (10  of  30)  and  31.7  percent  of  the  downtime.  In 
contrast  to  land-line  cables,  satellite  broadcasts  are  affected  by  inclement  weather.  This  problem 
can  be  alleviated  through  the  use  of  more  powerful  satellite  dishes  at  the  sites,  higher 
bandwidths,  or  software  upgrades  to  the  equipment.  Given  the  high  costs  associated  with  the 
first  two  optiors,  software  upgrades  by  the  contracter  have  been  the  principal  means  of  address¬ 
ing  this  problem. 

In  the  evaluation  period,  downtime  occurred  rather  infrequently,  averaging  about  1.4  incidents 
per  course  convening  (i.e.,  34  incidents/25  convenings).  Furthermore,  the  average  downtime  per 
incident  was  only  29  minutes  (i.e.,  985  minutes/34  incidents).  When  the  regression  model  dis¬ 
cussed  in  section  lil  is  expanded  to  include  downtime,  downtime  (however  defined)  has  no 
significant  effect  on  course  grades.  Not  only  was  downtime  a  relatively  minor  problem  during  the 
first  six  months  but  it  actually  declined.  Downtime  per  class-day  was  27  percent  lower  in  the 
second  quarter  of  the  evaluation  period  than  in  the  first  quarter.2 


1 .  Specific  reasons  for  each  incident  are  shown  in  appendix  E. 

2.  Downtime  during  die  first  two  quarters  of  the  evaluation  period  averaged  8.5  and  6.2  minutes  per  class- 
day,  respectively. 


SECTION  6 


ANALYSIS  OF  NET  SAVINGS 


This  section  examines  the  Navy’s  net  savings  from  using  VTT,  during  the  first  six  months. 
Gross  savings  are  defined  as  the  travel  and  per  diem  that  v/ould  have  been  incurred  in  sending 
personnel  from  the  remote  sites  to  the  originating  site.  For  the  VTT  courses,  this  involved 
sending  students  from  Charleston  and/or  Mayport  to  either  Dam  Neck  or  Norfolk.  For  the  other 
uses  of  the  system,  the  originating  site  was  defined  as  the  site  with  the  most  participants. 
Because  of  the  close  proximity  of  Dam  Neck  and  Norfolk,  their  attendance  was  combined  when 
selecting  the  originating  site  for  the  other  uses. 

Table  19  shows  that  the  gross  savings  from  travel  and  per  diern  was  $278,721.  Estimates  of 
the  cost  of  air  travel,  ground  travel,  and  per  diem  were  obtained  from  the  comptroller’s  office  at 
FCTCLANT.  One-half  of  a  day’s  travel  time  was  used  in  computing  the  per  diem.  Daily 
per  diem  is  higher  for  officers  and  civilians  than  for  enlisted  personnel  because  officers  and 
civilians  usually  stay  off-base  and  enlisted  personnel  generally  stay  on-base  at  the  Bachelors’ 
Enlisted  Quarters  (BEQ).  Car  rentals  were  assumed  for  those  staying  off-bast:. 


Table  18.  Summary  of  cost  analysis 


Gross  savings  in  travel 

ano  per  diem  (all  uses): 

$278,721 

Minus 

Cost  of  using  VTT  system: 

$210,000 

Equals 

Net  savings  from  VTT: 

$68,721 

Since  the  instructor’s  services  would  have  been  required  under  both  the  traditional  method 
and  VTT,  the  net  cost  of  using  instructors  was  assumed  to  be  zero.  !f  VTI  were  used  on  a  much 
larger  scale,  there  could  be  net  savings  from  training  the  same  number  of  students  with  fewer 
instructors.  Alternatively,  more  students  could  be  trained  with  the  same  number  of  instructors 
through  live  use  of  multiple  remote  sites. 

The  program  manager  and  on-site  facilitators  performed  collateral  duties  ir  addition  to 
VTI'  management.  Because  new  billets  were  not  created  during  this  initial  six  month  period,  the 
marginal  cost  of  using  their  services  was  assumed  to  be  zero.  If  VTT  were  implemented  on  a 
larger  scale,  die  analysis  would  need  to  include  die  cost  of  creating  new  billets  or  contracting  for 
die  appropriate  personnel  to  manage  the  VTI'  system. 


The  cost  of  using  VTT  was  estimated  to  be  the  amount  budgeted  for  the  six-month  period, 
$210,000.  Net  savings  were  then  computed  as  the  difference  between  between  gross  savings  and 
the  cost  of  using  VTT.  The  VTT  system  generated  net  savings  of  $68,721,  which  translates  into 
$17,180  per  site. 

This  estimate  of  net  savings  applies  specifically  to  the  six-month  evaluation  period.1  It 
should  not  be  used  to  predict  the  cost-effectiveness  of  any  future  system.  The  benefits  and  costs 
of  VTT  are  likely  to  increase  as  the  system  expands.  Whether  the  future  system  will  save  money 
depends  largely  on  the  rate  at  which  the  system  is  utilized.  Most  of  the  costs  of  the.  future  system 
(including  satellite  time)  will  likely  be  fixed  costs  that  do  not  vary  with  system  utilization. 
Greater  utilization  of  the  system  should  then  increase  die  benefits  by  reducing  the  travel  and 
per  diem  of  students.  A  more  detailed  analysis  of  the  costs  and  benefits  of  the  future  system  will 
be  a  subject  for  further  research. 

Table  20  shows  the  division  of  gross  savings  by  type  of  use.2  VTT  courses  used  the  system 
for  1 36  class-days  and  generated  gross  savings  of  $142,822.  In  contrast,  the  other  uses  employed 
the  system  for  only  16  days  and  generated  gross  savings  of  $135,899.  Gross  savings  per  day 
were  $1,050  for  VTT'  courses  and  $8,494  for  the  other  uses  of  the  system.  The  other  uses  were 
more  cost-effective  than  the  VTT  courses  because  their  convenings  were  much  shorter.  The 
average  length  of  a  convening  was  5.4  days  for  the  VTT  courses  and  1  day  for  the  other  uses. 


Tabla  20.  Gross  savings  by  type  of  use 


VTT 

courses 

Other  uses 

Military 

Civilian 

Number  of  days  in  use 

136 

9 

7 

Number  of  convenings 

25 

9 

7 

Gross  savings 

$142,822 

$58,313 

$77,586 

Gross  savings  per  day  in  use 

$1,050 

$6,479 

$11,084 

Days  in  use  p9r  convening 

5.4 

1.0 

1.0 

The  contractors  were  responsible  for  surveying  sues,  leasing  most  of  the  V'H'  equipment  including  live 
satellite  disk  providing  maintenance,  and  procuring  satellite  time. 

2.  Appendix  F  provides  a  breakdown  of  gross  savings  by  convening. 


During  this  period,  civilian  uses  accounted  for  more  savings  per  day  than  other  military 
uses.  Based  on  data  from  seven  convenings,  civilian  uses  generated  gross  savings  of  $1 1 ,084  per 
day.  In  contrast,  other  military  uses  generated  $6,479  per  day  based  on  data  from  nine  conven¬ 
ings.  The  principal  reason  for  this  difference  was  shown  in  table  2.  The  civilian  uses  averaged 
more  sites  per  convening  than  the  other  military  uses. 

Shorter  convenings  are  more  cost-effective  because  they  permit  more  personnel  to  use  tne 
system  over  any  given  period  of  time.  This  principle  not  only  is  illustrated  in  table  20  by 
comparing  other  uses  to  VTT  courses  but  is  also  demonstrated  by  comparing  VTT  courses  of 
different  length  in  table  21. 1  Two-day  courses  averaged  $1,906  per  day  in  gross  savings,  and 
20-day  courses  averaged  only  $525  per  day  in  gross  savings.  The  lower  gross  savings  for  the 
10-day  course  relative  to  the  20-day  courses  is  due  to  lower  attendance  at  remote  sites  in  the 
10-day  course  rather  than  course  length. 


Table  21.  Cost  avoidance  by  course  length 


Course  length 
in  days 

Number  of 
convenings 

Gross 

savings 

Gross  savings 
per  class  day 

2 

4 

$15,248 

$1,906 

3 

6 

31,462 

1,748 

5 

12 

70,906 

1,182 

10 

1 

4,214 

421 

20 

2 

20,991 

525 

1.  The  official  course  length  ,s  user)  in  this  table,  rather  than  the  actual  course  length.  In  most  eases,  they 
wen'  tire  same.  However,  the  actual  length  could  liavc  been  shorter  due  u>  a  holiday  during  the  week. 


SECTION  7 


CONCLUSIONS  AND  RECOMMENDATIONS 


Course  grades  were  lower,  on  average,  under  VTT  than  under  traditional  methods  of 
instruction.  The  differences  in  grades  was  small  enough  that  failure  rates  were  low  and  did  not 
differ  significantly  between  the  two  training  methods.  If  VTT  were  used  in  courses  with  much 
lower  mean  grades,  failure  rates  might  become  much  higher  under  the  VTT  method  than  under 
the  traditional  method  of  instruction.  Increasing  the  number  of  remote  sites  from  one  to  two  for 
a  given  course  also  reduced  course  grades.  Whether  failure  rates  would  change  dramatically  with 
a  further  increase  in  the  number  of  sites  is  an  unresolved  question.  Further  research  on  this  issue 
requires  convenings  of  courses  that  both  are  graded  and  use  three  or  more  remote  sites. 

A  survey  of  students  identified  three  main  areas  of  deficiency  in  the  VTT  method  of 
instruction:  the  quality  of  the  video,  the  level  of  instructor-student  interaction,  and  the  quality  of 
the  audio.  Improving  the  quality  of  the  audio  and  video  transmissions  is  primarily  a  technical 
issue.  During  the  latter  part  of  the  evaluation  period,  efforts  were  made  to  improve  VTT  by 
increasing  the  size  of  the  television  monitors,  adding  additional  microphones  for  the  students, 
using  a  cordless  microphone  for  the  instructor,  and  incorporating  new  software  in  video 
compression/decompression.  Data  should  be  collected  during  tire  next  year  to  determine  the 
efficacy  of  these  changes. 

If  instructor-student  'nteraction  is  to  be  improved,  greater  attention  needs  to  be  focused  on 
how  instructors  arc  trained  in  using  the  VTT  method  as  well  as  modifying  or  redesigning  course 
materials  to  more  effectively  fit  tire  medium.  The  Navy’s  proposed  research  laboratory  for  VTT 
could  play  an  important  role  in  addressing  these  issues.  In  particular,  it  would  be  very  useful  to 
retrain  instructors  and  change  course  materials  for  those  instructors  and  courses  that  have  used 
VTT  during  the  first  six  mondis.  Course  grades  and  surveys  of  boih  students  and  instructors 
could  then  assess  the  relative  improvement  in  VTT  training  effectiveness  from  applying  the 
lessons  learned  in  a  laboratory  environment  to  an  actual  teaching  environment. 

T  he  cost-el  fectivencss  of  VTT  wifi  depend  upon  the  extent  to  which  Lhe  system  is  utilized. 
During  the  first  six  months,  less  than  half  of  all  classrooms  were  used  on  a  typical  weekday. 
Benefits  should  increase  as  scheduling  improves.  In  addition,  shorter  courses  should  be  selected 
with  higher  throughput  at  multiple  remote  sites.  The  "other  uses”  of  the  system,  which  generated 
a  greater  than  proportionate  amount  of  gross  savings,  should  be  expanded  given  the  excess 
capacity  within  the  system.  Costs  also  will  increase  as  die  system  expands.  In  the  long  run, 
advances  in  technology  should  lower  costs  and  improve  die  quality  of  the  medium.  Further 
cosl-benelil  analyses  will  tv  required  as  additional  data  become  available. 
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VTT  COURSE  CONVENINGS 


During  the  evaluation  period  of  the  Navy’s  VTT  network,  a  total  of  705  students  were  trained  in 
25  convenings  of  'he  12  VTT  courses.  Tables  A-l  and  A  2  provide  detailed  breakdowns  of 
attendance  by  convening  and  site. 


Yabn»  A-1.  VTT  course  convenings  (1st  quarter) 


Course 

Week 

Dam  Nock 

Number  of  students  by  site 

Charleston  Mayport 

Norfolk 

Total 

1.  Opsec.  Plan. 

1 

0a 

0 

9 

0 

9 

2.  Track  Sup. 

2 

0* 

13 

3 

0 

16 

3.  Gun  Bat.  Align. 

2 

8a 

9 

0 

0 

17 

4.  Ammc.  Admin. 

3 

22a 

14 

13 

12 

61 

5.  Advanced  Sig. 

4 

0 

6 

e 

1 4a 

28 

6.  Adv.  Com.  Proc. 

6 

C 

4 

0 

I4a 

18 

7.  CCCC 

8 

0 

0 

17 

18a 

35 

8.  Ammo.  Admin. 

11 

26a 

28 

c 

0 

i>4 

9.  Adv.  Com.  Proc. 

13 

0 

15 

0 

17a 

32 

10.  Track  Sup. 

13 

1 6a 

3 

1 1 

0 

30 

Total 

72 

92 

51 

75 

300 

a.  Indicates  criginating  site. 


Tabl®  A-2.  VTT  course  convenings  (2nd  quarter) 


Number  cf  students  by  site 


Course 

Week 

Dam  Nock 

Charleston 

Mayport 

Norfolk 

Total 

11. 

Th.  Nuc.  War 

14 

0® 

0 

3 

0 

3 

12. 

SAS/EAP 

14 

0a 

8 

0 

0 

3 

13. 

Sov.  Sig.  Proc. 

15 

0 

11 

9 

14a 

34 

14. 

Gun  Bat.  Align. 

17 

11’ 

0 

n 

O 

yj 

A 

1  » 

IS. 

Opsec  Plan. 

18 

20s 

6 

0 

0 

28 

16. 

GCCC 

19 

0 

17 

0 

20a 

37 

17. 

SAS/EAP 

19 

o*1 

0 

12 

0 

12 

18. 

Track  Sup. 

20 

19a 

12 

3 

0 

34 

19. 

Opsec.  Plan. 

21 

22a 

6 

10 

0 

38 

20. 

Ammo.  Admin. 

22 

25a 

13 

18 

0 

56 

21. 

Track  Sup. 

23 

1 7n 

15 

13 

0 

45 

22. 

Sov.  Sig.  Proc. 

24 

0 

9 

0 

20a 

29 

23. 

Adv.  Com.  Proc. 

25 

0 

9 

0 

18a 

27 

24. 

50  Cal.  MG  Maint. 

25 

oa 

8 

12 

0 

20 

25. 

NW  Rad.  Cert. 

27 

0a 

15 

0 

0 

15 

Total 

114 

131 

88 

72 

405 

a.  Indicates  originating  site. 
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REGRESSION  RESULTS  FROM  ALTERNATIVE  SPECIFICATIONS 


A  regression  model  was  used  to  evaluate  the  relative  effectiveness  ct  using  VTT  in 
Navy  training  courses.  Tables  8-1  through  B-3  document  tire  results  from  alternative  specifi¬ 
cations  of  the  model. 


Table  B-1.  Additional  variables  in  alternative  specifications* 


Variable 


Definition 


Control  group 


CONV(r) 


1 :  student  in  /th  course  convening  Students  in  third  course  convening 
0:  otherwise 


ORIG(y) 


1 :  student  at  originating  site  and  Students  at  originating  site  when  convening 
convening  has  total  of  j  sites  has  a  total  ot  two  sites 
0:  otherwise 


REMO  TE(k) 


1 :  student  at  remote  site  and 
convening  has  total  of  /(sites 
0.  otherwise 


Students  at  originating  site  when  convening 
has  total  of  two  sites 


a.  Table  5  contains  definitions  for  those  variables  that  were  also  used  in  the  main  regression  model. 


B-1 


Table  B-2.  Regression  results  with  1 2  convening-specific  dummy  variables 
(dependent  variable:  GRADE) 

Independent 

variable 

Mean 

(standard  deviation) 

Coefficient 
(t  statistic) 

Intercept 

81. 1a 

(38.2) 

REMOTE 

.48 

-2.3a 

(.50) 

(-3.5) 

AFQT 

62.1 

.15a 

(18.3) 

(7.6) 

YOS 

6.0 

,33a 

(4.4) 

(3.3) 

CONV(5) 

.06 

-  6.3a 

(.24) 

(-3.1) 

CONV(6) 

.04 

-1.7 

(.20) 

(-.8) 

CONV(7) 

.09 

-3.1 

(.25) 

(-1.6) 

CONV(9) 

.07 

-5.4a 

(.26) 

(-2.7) 

CCNV(IO) 

.08 

-6.1a 

(.27) 

(-3.2) 

conv(i  : 

.10 

-6.33 

(.30) 

(-3.4) 

CONV(H) 

.05 

-1.9 

(.21) 

(-9) 

CONV(16) 

.10 

-2.1 

(.30) 

(-1.1) 

CONV(18) 

.10 

-4.5b 

(.30) 

(-2.4) 

OGNV(21) 

.12 

-s.o3 

(.32) 

(-2-8) 

CONV(22) 

.06 

-4.1 b 

(.2 '/) 

(-2.1) 

CONV(23) 

.06 

-3.9b 

(.25) 

(-1.9) 

R-sguare 

.34 

F-statistic 

11. 2° 

Sample  size 

341.0 

NOTE:  Mean  and  standard  deviation  of  GRADE  are  87  2  and  7  3,  respectively. 

a  Statistically  significant  at  the  1  percent  level, 
b.  Statistically  significant  at  the  5  percent  level. 


Table  B-3.  Regression  results  with  interaction  of  site  type  by  number  of  sites 
(dependent  variable:  GRADE) 


Independent  Mean  Coefficient 

variable  (standard  deviation)  (t-statistic) 


Intercept 

80. 3a 

(43.4) 

ORIG(3) 

.21 

-2.2 

(.40) 

(-1-6) 

REMOTER) 

.27 

-2.4a 

(.44) 

(-2.8) 

REMOTER) 

.23 

-4.6a 

(.42) 

(-3.5) 

AFQT 

62.2 

,15a 

(18.1) 

(7.7) 

YOS 

5.9 

,34a 

(4.4) 

(3.4) 

COURSE1 

.33 

-2.0 

(.47) 

(-1.2) 

COURSE2 

.06 

-3.1 

(-24) 

(-1.5) 

COURSE3 

.17 

-3.1 b 

(.37) 

(-2.3) 

COURSE4 

.19 

-1.7 

(.39) 

(-1.2) 

COURSE5 

.17 

-3.2b 

(.37) 

(-2.1) 

R-square 

.33 

F-statistic 

16.6a 

Sample  size 

356.0 

NOTE:  Mean  and  standard  deviation  of  GRADE  are  87.2  and  7.2,  respectively. 

a.  Statistically  significant  at  the  1 -percent  level. 

b.  Statistically  significant  at  the  5-percent  level. 
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STUDENT  QUESTIONNAIRES 


This  appendix  contains  copies  of  the  student  questionnaires  used  in  evaluating  the  Navy’s 
VTT  courses.  The  questionnaires  were  developed  by  CNA  based  on  input  from  the  fleet  and 
other  organizations  represented  on  the  Steering  Committee. 


BACKGROUND 
QUESTIONNAIRE  FOR 
VTC  STUDENTS 


INSTRUCTOR:  Admmiittr  this  questionnaire  on  the 

first  day  of  clam  and  send  all  question*  | 
mire*  to  Dam  Neck  P.O.C. 

FACILITATOR:  Collect  questionnaires  and  send  to 

instructor,  who  wilt  forward  them.  1 


1.  Name _ _ 

Fir* 


2.SSN 


3.  Gender  G  a.  Mal«*  4.  Paygrade _ - _ 5.  Rating 

G  b.  Female 


6.  Today's  date _ / _ / 


7.  Course  title 


□  a.  Ammo  Admin 

□  b.  Gun  Bart  Align 

□  c.  TADIL-A  FORTRCOO 

□  d.  OPSEC  Planning 

□  e.  Advanced  SM 
C  f.  Soviet  Signaling 


D  g.  Comm  Procedures 

□  h.  CCCC 

□  i.  Intro  to  HWS 

Q  j.  Sub  Off  Mine  war 
O  k.  Other  (specify) _ 


8.  Type  of  classroom 

□  a.  Originating  where  instructor  is  physically  present 

□  b.  Remote  where  instructor  is  seen  on  television 


9.  Location  of  your  classcoom 

□  a.  FCTCLANT,  Dam  Neck,  VA 

□  b.  FTC,  Norfolk,  VA 

□  c.  FTC,  Mayport,  FL 

□  d.  FMWTC,  Charleston,  SC 

□  e.  Other  (specify) _ 

10.  City  and  state  of  your  duty  station _ ,  _ 

11.  Name  of  vour  command  (example:  FFC  36) _ 

12.  Axe  you  receiving  per  diem  while  taking  this  course? 

□  a.  Yes 

□  b.  No 

AN5WEB  QUESTIONS  13  THBOUCH  15  ONLY  If  YOU  ANSWEBED  YES  TO  QUESTION  12. 

13.  Type  of  berthing  when  taking  course 

□  a.  BEQ  or  BOQ 

D  b.  Commercial  hotel  or  motel 

□  c.  Other  (specify) _ _ _ 

14.  Mode  of  transportation  (from  dty  of  duty  station  to  course) 

□  a.  Air 
Li  b.  Bus 

□  c.  Tram 
G  d.  Auto 

G  e.  Other  (specify) _  _ _ 

15.  One-way  travel  ume  m  hours  (from  city  of  duty  station  to  course) _ 

VTC  fonm  1 

p*g«  i  of  1 


C  2 


COURSE 

QUESTIONNAIRE 
FOR  VTC  STUDENTS 


INSTRUCTOR:  Administer  lo  students  at  the  end  of  lost 
class  and  send  to  Darn  Neck  P.O-C. 

Facilitator:  Collect  questionnaires  and  send  to 
instructor,  who  will  forward  them. 


X.  Name 


1  Course  title 


Fin 

1  ifl 

La* 

_/_ 

J _ 

□  a. 

Ammo  Admin 

G  g.  Comm  Procedures 

□  b. 

Gun  Batt  Align 

□  h.  CCCC 

□  c. 

TADIL-A  FORTRCOO 

□  i.  Intro  to  HWS 

□  d. 

OPSEC  Planning 

□  j.  Sub  Off  Minewar 

□  e. 

Advanced  SM 

U  L  Other  (specify) _ 

□  f. 

Soviet  Signaling 

2.  SSN. 


Section  1:  Course  Evaluation _ _ 

For  each  of  the  following  statements  (1  through  21),  check  the  appropriate  box  corresponding  to  a  scale 
of  1  (unsatisfactory)  through  5  (outstanding).  Leave  any  statements  that  do  not  apply  to  this 
course  blank. 

UNi*mrAcrc*Y  — - - Outjtanoinc 


Instructor 

0-50% 

or  nm 

51-40% 

or  nui 

61-40% 

or  timr 

51-90% 

or  TIW! 

91-100% 

or  tivi 

1.  Instructor  prepared  for  class 

E 

□ 

E 

□ 

D 

Instructor  presented  lessons  clearly 

□ 

□ 

□ 

□ 

□ 

3.  Instructor  answered  student  questions 

□ 

□ 

□ 

□ 

E 

4.  Instructor  encouraged  class  partidpadon 

□ 

□ 

□ 

□ 

□ 

5.  Instructor  was  available  for  individual  assistance  outside  of  class 

E 

□ 

□ 

n 

□ 

6.  Instructor  treated  students  fairly 

E 

□ 

□ 

E 

LE 

Audio-Visual  Aids 

7.  Video  screen  was  large  enough  to  be  seen 

□ 

□ 

□ 

E 

□ 

$.  Video  screen  was  dose  enough  to  be  seen 

□ 

□ 

□ 

□ 

□ 

9.  Image  on  video  screen  was  dear 

n 

□ 

□ 

□ 

□ 

10.  Audio  transmission  was  loud  enough  to  hear  instructor  s  voice 

□ 

□ 

□ 

□ 

□ 

11.  Audio  dansmission  was  dear  enough  to  hear  what  instructor  said 

n 

□ 

E 

□ 

r*"1 

Lj 

12.  Graphics/Slides/Transparencies  on  TV  were  readable 

□ 

□ 

□ 

□ 

□ 

13.  Television  was  in  working  order 

□ 

□ 

n 

□ 

L _ J 

li.  Your  microphone  was  in  working  order 

□ 

□ 

□ 

□ 

VTC  fcrm  2  (Ovi's) 
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Tests/Homeworx 

0-50% 

51-60% 

61-80% 

91-90% 

91-100 

15.  Test  questions  were  dearly  written 

□ 

□ 

OP  TUMI 

□ 

OP  T1MI 

□ 

□ 

16.  Test  questions  were  directly  related  to  course 

□ 

□ 

□ 

□ 

□ 

17.  Test  answers  were  graded  fairly 

□ 

□ 

□ 

□ 

i  ' 

18.  Homework  assignments  were  understandable 

□ 

□ 

□ 

□ 

□ 

19.  Homework  assignments  were  directly  related  to  course 

□ 

□ 

□ 

□ 

□ 

Overall 

Unsatihactoiy 

-  Outstanding 

20.  Comparison  of  this  instructor  to  other  Navy  instructors  that 

□ 

□ 

□ 

□ 

□ 

have  taught  you  in  the  past 

21.  Comparison  of  this  course  to  other  Navy  courses  that  you  have 

□ 

□ 

□ 

□ 

□ 

taken  in  the  past 

Section  2:  Instructor-Student  Interaction 


22.  Did  you  talk  to  the  instructor  or  ask  any  questions  during  the  regular  hours  of  this  course? 

□  a.  Yes 

□  b.  No 

23.  How  did  the  video  tele-training  method  of  instruction  affect  your  opportunities  to  talk  to  the  instructor  or  ask 
questions,  as  compared  to  traditional  methods  of  instruction? 

□  a.  More  opportunities 

□  b.  No  effect  or.  opportunities 

□  c.  Fewer  opportunities 

24.  Were  there  adequate  opportunities  for  remedial  instruction  outside  of  the  regular  hours  of  this  course? 

□  a.  Yes 
U  b.  No 

□  c.  Remedial  instruction  was  not  necessary  for  this  course. 


25.  Did  you  attmd  any  remedial  instruction  periods? 

n  a.  Yes 

□  b.  No 


Am  WEI  QUESTIONS  26  AND  27  ONLY  IF  YOU  ANSWERED  YES  TO  QUESTION  25. 


26  From  whom  did  you  obtain  the  remedial  instruction? 

Lj  a.  Instructor  va  video  tele  training 
C  b.  Instructor  who  was  physically  present  in  same  room 
□  c.  Other  (specify) _ _ _ _ _ _ _ 

27.  How  many  hours  of  remedial  instruction  did  you  receive? _ 

VTC  fonn  2 
nag*  2ol  J 
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Section  3:  Student  Comments 


28.  Which  method  of  instruction  would  you  have  preferred  for  this  course? 

~  a.  Video  tele-training  where  instructor  is  on  TV 

□  b.  Traditional  methods  of  instruction  where  instructor  is  physically  present  in  the  classroom 
C]  c.  Indifferent  between  video  tele-training  and  traditional  methods  of  instruction 

29.  How  did  the  participation  of  students  at  other  sitefs)  affect  your  learning  during  this  course? 

□  a.  Improved  learning 

□  b.  No  effect  on  learning 

□  c.  Reduced  learning 

30.  What  did  you  like  most  about  this  course? 


31.  What  did  you  like  least  about  this  course? 


32.  Discuss  any  suggestions  that  you  have  for  improving  how  video  tele-training  is  used  in  this  course. 
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ADDITIONAL  STATISTICS  FROM  STUDENT  COURSE  EVALUATIONS 


Tables  D-l  through  D-6  support  section  4’s  discussion  of  the  student  evaluation  of 
Navy  VTT.  The  tables  show  sample  sizes  by  type  of  site  and  actual  t-statistics  for  each  item  and 
for  the  overall  evaluation. 


Table  D-1.  Student  evaluation  of  instructor 


Statement 

Instructor  prepared  tor  class. 

Instructor  presented  lessons  clearly. 

Instructor  answered  student  questions. 

Instructor  encouraged  class  participation. 

Instructor  was  available  for  individual  assistance 
outside  of  class. 


Sample  size 

t-statistic  for 
difference 

in  means 

Remote 

Originating 

332 

316 

-6.5 

332 

315 

-10.4 

331 

314 

-6.5 

332 

314 

-6.8 

282 

312 

-15.9 

321 

315 

-6.3 

Instructor  treat od  students  fairly. 


Table  D-2.  Stueeni  evaluation  of  audio-visual  aids 


Sample  size 

t-statisiic  for 

Statement 

Remote; 

Originating 

difference 
in  means 

Video  screen  was  large  enough  to  be  seen. 

331 

255 

. 4.0 

Video  screen  was  close  enough  to  be  seen. 

328 

257 

~3  3 

Image  on  video  screen  was  dear. 

331 

257 

-8.5 

Audio  transmission  was  ioud  enough  to  hear 
instructor’s  voice. 

331 

222 

-5.9 

Audio  'transmission  was  -tear  enough  to  hear 
•'••hat  iristructor  said. 

330 

21S 

-8.G 

Graphics/slides/transparencias  on  TV 
were  readable. 

327 

249 

-14.0 

Te'evision  was  in  working  order. 

330 

257 

-1.1 

Your  microphone  was  in  working  order. 

32£ 

236 

-1.5 

Table  D-3.  Student  evaluation  of  tests  and  homework 


Statement 

Remote 

Sample  size 

Originating 

t -statistic  for 
difference 
in  means 

Tost  questions  wera  clearly  written. 

290 

264 

-4,5 

Test  questions  were  redly  related  to  course. 

289 

264 

-5.7 

Test  answers  were  graded  'airly. 

269 

259 

4.3 

Homework  assignments  were  under  star.dab’e 

24S 

224 

-3.1 

Homework,  assignments  were  directly  related 
to  eouise 

246 

227 

-2.8 

D-2 


Table  D-4.  Student  overall  evaluation  of  VTT  course 


I 


Sample  size 

t-stati&tic  for 

Statement 

Remote 

Originating 

Difference 
in  percent 

Comparison  of  this  instructor  to  other  Navy 
instructors  that  have  taught  you  in  the  past. 

328 

307 

-9.8 

Comparison  of  this  course  to  other  Navy  courses 
that  you  have  taken  in  the  past 

329 

307 

-1 1 .3 

Table  0-5.  Student  responses  to  other  items  on  course  questionnaire 

Statement 

Remote 

(percent) 

Originating 

(percent) 

t-statistic  for 
difference 
in  percent 

Preferred  traditional  method  of  instruction 

6-4.2 

78.2 

-3.9 

Interacted  with  instructor  during  regular  hours 
of  course 

76.6 

92.4 

-6.0 

Perceived  fewer  opportunities  to  interact  with 
instructor  using  VTi  .  .iative  to  traditional 
methods  of  instruction 

55.0 

15.1 

11.5 

Attended  any  remedial  instruction  periods 

13.4 

21.8 

-2.8 

Perceived  that  opportunities  for  remedial 
instruction  were  adequate 

41.6 

73.2 

-  8.5 

Peiceived  that  participation  of  students  at 

18.2 

1 7.5 

0.2 

other  sires  reduced  own  learning 


Tab!®  D-6.  S-imple  size  for  other  items  or.  course  questionnaire 


Korn 

Pemoto 

Originating 

Preference  as  to  method  cf  iristru^iun 

327 

333 

interaction  with  instructor  during  regular  nours  of  couma 

332 

nig 

Opportunities  to  interact  wiih  instructor  using  VTT  relative  to 
aditional  methods  of  instruction 

331 

235 

Attendance  at  remedial  instruction  ceriods 

326 

307 

Adequacy  of  opportunities  for  remedial  instruction 

329 

310 

Impact  of  participation  of  students  at  other  sites  on  ow^  learning 

313 

309 

D  4 
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SPECIFIC  REASONS  FOR  DOWNTIME  BV  INCIDENT 


Tables  E-!  and  E-2  show  the  amounts  of,  and  specific  reasons  for,  downtime  in  each 
convening  of  the  Navy's  VTT  courses  during  the  first  six  months. 


labia  E*1.  Reasons  for  downtime  (1st  quarter) 


Convening 

nu.T.bcr 

Downtime 

(minutes) 

Source  of  problem 

1 

120 

NASA  launch  delays  start 

2 

30 

Bad  weather  at  Darn  Neck 

2 

0* 

Modem  at  Mayport 

3 

40 

Satellite  dish 

4 

45 

Disk  interfacing  with  CODEC 

4 

15 

Bad  weather  at  Dam  Neck 

5 

30 

Charleston  audio 

6 

15 

Loose  cable  n  CODEC  link 

7 

in 

Microphone  at  Norfolk  and  switching  problems 
at  Dam  Neck 

7 

10 

Power  outage  at  Norfolk 

7 

60 

Bad  wea’har  at  Mayport 

7  &  8 

10 

Satellite  carrier 

9  &  10 

15 

Brown-out 

9  &  10 

30 

Bad  weather  at  Charleston 

10 

o3 

Power  outage  at  Mayport  due  to  weather 

;o 

10 

Bad  weather  at  Mayport 

10 

90 

Bad  weather  at  Mayport 

a.  Students  worn  dismi; 

isud  oar!y. 

E 


0, 


Table  E-2.  Reasons  fordowntime  (2nd  quarter) 


Convening 

number 

Downtime 

(minutes) 

Source  of  problem 

13 

0 

Modem  at  Charleston  1 ;  switched  classrooms 

14 

4 

Bad  weather  at  Mayport 

15 

90 

Resister  in  power  supply  at  Charleston 

16 

15 

Bad  weather 

16 

45 

Technical  difficulties 

16  &  18 

30 

Bad  weather  at  Charleston 

16 

45 

Power  outage  at  Norfolk 

20 

16 

Bad  weather 

21 

40 

Torn  plastic  cover  on  earth  station 

21 

25 

Audio  problems  at  Charleston  and  Mayport 

21 

15 

Audio  problems  at  Charleston 

22 

15 

Technical  difficulties 

23 

30 

Loose  cable  on  modem  at  Charleston 
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APPENDIX  F 


COST  AVOIDANCE  BY  SPECIFIC  USE 


Tables  F-l  through  F-4  support  section  6’s  discussion  of  the  Navy’s  savings  from  using 
VTT,  during  the  first  six  months.  Tables  F-l  and  F-2  show  the  gross  savings  from  using  VTT  for 
Navy  course  training.  Tables  F-3  and  F-4  show  the  gross  savings  from  using  VTT  for  other  uses, 
such  as  briefings  on  training  exercises,  civilian  training,  and  conferences. 


Table  F-1.  Cost  avoidance  by  course  convening  (1st  quarter) 


Convening 

number 

Number  who  would 
have  traveled 

Travel  savings 
(round-trip) 

Per  diem 
savings 

Gross 

savings 

1 

9 

$2,537 

$1,444 

$3,981 

2 

16 

4,101 

466 

4,567 

3 

9 

2,349 

262 

2,611 

4 

27 

7,842 

4,118 

1 1 .959 

5 

14 

3,286 

928 

4,214 

6 

4 

996 

117 

1,113 

7 

17 

3,808 

2,388 

6,196 

8 

28 

7,728 

2,065 

9,793 

9 

15 

3,735 

437 

4,172 

10 

14 

3,379 

408 

3,787 

Total 

153 

$39,761 

$12,633 

$52,393 

F-l 


Table  F-2.  Cost  avoidance  by  course  convening  (2nd  quarter) 


Convening 

number 

Number  who  would 
have  traveled 

Travel  savings 
(round-trip) 

Per  diem 
savings 

Gross 

savings 

11 

3 

$757 

$206 

$963 

12 

8 

2,604 

1,738 

4,342 

13 

20 

4,755 

371 

5,126 

14 

8 

2,001 

531 

2,532 

15 

8 

2,324 

863 

3,187 

16 

17 

6,390 

8,406 

14,796 

17 

12 

3,027 

824 

3,851 

18 

15 

3,840 

437 

4,277 

19 

16 

4,634 

2,483 

7,117 

20 

31 

9,181 

5,484 

14,665 

21 

28 

7,123 

1,233 

8,356 

22 

9 

2,339 

432 

2,771 

23 

9 

2,393 

679 

3,072 

24 

20 

5,245 

1,373 

6,618 

25 

15 

5,033 

3,723 

8,756 

Total 

203 

$61,646 

$28,783 

$90,429 

Table  F-3.  Cost  avoidance  by  other  use  (1st  quarter) 


Other  use 

Number  who  would 
have  traveled 

Travel  savings 
(round-trip) 

Per  diem 
savings 

Gross 

savings 

CINTEX 

27 

$7,561 

$3,281 

$10,842 

FLEETEX 

25 

5,836 

3,038 

8,874 

OCPM  conf. 

37 

10,441 

4,496 

14,937 

OCPM  conf. 

22 

6,196 

2,673 

8,869 

OCPM  conf. 

40 

11,370 

4,860 

16,230 

OCPM  training 

17 

4,781 

2,066 

6,847 

OCPM  training 

25 

7,000 

3,038 

10,038 

OCPM  training 

16 

4,563 

1,944 

6,507 

OCPM  training 

35 

9,905 

4,253 

14,158 

TCCTconf. 

14 

3,752 

1,701 

5,453 

Total 

258 

$71,405 

$31,350 

$102,755 

■< 


r 


F-2 


Table  F-4.  Cost  avoidance  by  other  use  (2nd  quarter) 


Other  use 

Number  who  would 
have  traveled 

Travel  savings 
(round-trip) 

Per  diem 
savings 

Gross 

savings 

BFIT 

5 

$1,340 

$-508 

$1,948 

BFIT 

13 

3,484 

1,580 

5,064 

BFIT 

<>0 

5,360 

2,430 

7,790 

BFIT 

10 

2,680 

1,215 

3,895 

BFIT 

12 

3,216 

1,458 

4,674 

CINTEX 

24 

6,857 

2,916 

9,773 

Total 

84 

622,937 

$10,207 

$23,144 

F-3 


